Acquisition, recovery, and matching of unique information from file-based media for automated file detection

ABSTRACT

A media fingerprint archive system generates and archives media fingerprints from second media content portions such as commercials. A downstream media measurement system can extract/derive query fingerprints from an incoming signal, and query the media fingerprint archive system whether any of the query fingerprints matches any archived fingerprints. If so, the media measurement system can perform media measurements on a specific secondary media content portion from which the matched query fingerprint is derived. If not, the media measurement system can analyze media characteristics of a media content portion to determine whether the media content portion is a secondary media content portion and perform media measurement if needed to. The media measurement system may send fingerprints from an identified secondary media content portion to the media fingerprint archive system for storage.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/781,549, filed 30 Sep. 2015, which claims priority to PCT ApplicationNo. PCT/US14/25979, filed on 13 Mar. 2014, which claims priority to U.S.Provisional Patent Application No. 61/809,001, filed on 5 Apr. 2013. Theentire contents of the foregoing applications are hereby incorporated byreference as if fully set forth herein. The applicant(s) hereby rescindany disclaimer of claim scope in the parent application(s) or theprosecution history thereof and advise the USPTO that the claims in thisapplication may be broader than any claim in the parent application(s).

TECHNOLOGY

The present invention relates generally to media processing systems, andin particular, to acquisition, recovery, and matching of uniqueinformation from file-based media for automated file detection.

BACKGROUND

Broadcast networks and operators depend on commercials for revenue.However, commercials are produced and provided by many differententities and individuals. A commercial embedding within non-commercialnormal program materials may have objectionable characteristics such asunreasonable loudness to many viewers/listeners. Given potentially avery large audience for its programs, a broadcast network or operatormay receive a large number of complaints to commercials withobjectionable characteristics that are in the programs.

To address these complaints as required by law and to comply withrelevant viewer protection regulations, a broadcast network or operatormay need to deploy significant human resources in monitoring itsbroadcast channels or media data bitstreams which the broadcast networkor operator receives or sends/broadcasts. The information gatheredthrough monitoring may be used to determine whether these channels orbitstreams contain commercials with objectionable characteristics inresponse to complaints or audits. It is questionable whether theproblems with these commercials can be effectively identified, even whenthe broadcast network or operator spends a large amount of resources onmonitoring its programs.

The approaches described in this section are approaches that could bepursued, but not necessarily approaches that have been previouslyconceived or pursued. Therefore, unless otherwise indicated, it shouldnot be assumed that any of the approaches described in this sectionqualify as prior art merely by virtue of their inclusion in thissection. Similarly, issues identified with respect to one or moreapproaches should not assume to have been recognized in any prior art onthe basis of this section, unless otherwise indicated.

BRIEF DESCRIPTION OF DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements and in which:

FIG. 1 shows a media fingerprint archive system;

FIG. 2 shows a media measurement system;

FIG. 3 illustrates an example algorithm or process flow to search forand perform measurements on secondary media content portions in mediadata;

FIG. 4A and FIG. 4B illustrate example process flows; and

FIG. 5 illustrates an example hardware platform on which a computer or acomputing device as described herein may be implemented.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Example embodiments, which relate to acquisition, recovery, and matchingof unique information from file-based media for automated filedetection, are described herein. In the following description, for thepurposes of explanation, numerous specific details are set forth inorder to provide a thorough understanding of the present invention. Itwill be apparent, however, that the present invention may be practicedwithout these specific details. In other instances, well-knownstructures and devices are not described in exhaustive detail, in orderto avoid unnecessarily occluding, obscuring, or obfuscating the presentinvention.

Example embodiments are described herein according to the followingoutline:

-   -   1. GENERAL OVERVIEW    -   2. MEDIA DATA AND MEDIA CONTENT PORTIONS    -   3. MEDIA FINGERPRINT ARCHIVE SYSTEM    -   4. MEDIA MEASUREMENT SYSTEM    -   5. ALGORITHM TO DETECT SECONDARY CONTENT PORTIONS    -   6. EXAMPLE PROCESS FLOW    -   7. IMPLEMENTATION MECHANISMS—HARDWARE OVERVIEW    -   8. EQUIVALENTS, EXTENSIONS, ALTERNATIVES AND MISCELLANEOUS

1. GENERAL OVERVIEW

This overview presents a basic description of some aspects of anembodiment of the present invention. It should be noted that thisoverview is not an extensive or exhaustive summary of aspects of theembodiment. Moreover, it should be noted that this overview is notintended to be understood as identifying any particularly significantaspects or elements of the embodiment, nor as delineating any scope ofthe embodiment in particular, nor the invention in general. Thisoverview merely presents some concepts that relate to the exampleembodiment in a condensed and simplified format, and should beunderstood as merely a conceptual prelude to a more detailed descriptionof example embodiments that follows below.

In some embodiments, a media fingerprint archive system with afingerprint engine or a server-side fingerprint derivation unit—asoperated by a commercial aggregator, a media network, a broadcastcompany, a large network operator, etc.—processes secondary mediacontent portions to be embedded with primary media content portions inmedia data in downstream media systems; generates media fingerprintsfrom media data units of the second media content portions; and storesthe media fingerprint in a fingerprint archive.

As used herein, a fingerprint archive may be an online, cloud-baseddatabase, a proprietary database, etc. A “media data unit” may comprisemedia sample data for a time unit. Examples of media data units may be aplurality of contiguous media samples or frames for a time unit of 32milliseconds, 64 milliseconds, etc. “Secondary media content portions”may be commercial advertisements, promotional advertisements, etc., invarious types of media. “Primary media content portions” may beprograms, shows, game broadcasts, concerts, movies, etc., in varioustypes of media. “Media data” may comprise primary media content portionsembedded with secondary media content portions, for example, inallocated advertisement time slots. Examples of media include but arenot limited to: audio only, video only, audio video, media files, mediabitstreams, etc. Other information including but not limited to otherunique identifiers other than the media fingerprints, sequentiallyassigned identifiers for corresponding secondary media content portions,unique alphanumeric labels for corresponding secondary media contentportions, etc., in addition to the media fingerprints can also bestored, archived and made accessible with the media fingerprints in thefingerprint archives.

The media fingerprint archive system may be configured to support one ormore (downstream) media measurement systems. In some embodiments, afingerprint engine or a client-side fingerprint derivation unit of adownstream media measurement system can extract/derive/generate mediafingerprints from an incoming signal, bitstream, file, etc., inreal-time, near-real-time, off-line, non-real-time, etc.

In an example embodiment, a media measurement system—as operated by anetwork operator or station, etc.—processes media data to bebroadcasted, downloaded or otherwise distributed to viewers/listeners;generates query fingerprints from media data units of the media data;sends fingerprint identification queries to the media fingerprintarchive system; and receives fingerprint identification responses fromthe media fingerprint archive system to determine whether any of thequery fingerprints corresponds to a secondary media content portion inthe media data.

In response to a positive fingerprint identification response thatindicates the presence of a specific secondary media content portion, anindividual set of media measurements including but not limited toloudness measurements, etc., can be performed for the specific secondarymedia content portion.

If a fingerprint match is not readily found between a query fingerprintand archived fingerprints, media characteristics and otheridentification information within the media data can be analyzed. Forexample, the downstream media measurement system can analyze time ofday, duration, frequency of scene changes, facial recognition,figure/form recognition, presence of speech, presence of music,identification of audio transients, visual spectral content, audiospectral content, video effects, ticklers, pre-existing secondaryprogram identifiers, network watermarks, rating watermarks,captioned/embedded texts, applause, program and scene transition,loudness, etc., in the media data. Classifiers or classificationalgorithms can be used to determine whether a particular media contentportion or a media data unit is or is not a secondary media contentportion, using weighting based on the presence or absence of suchclassifiers. For the purpose of the invention, it should be noted thatother media characteristics other than those enumerated herein may beused by classifiers or classification algorithms to determine mediacontent types. An analysis of a large amount of “secondary mediacontent” (e.g., training data, labeled data, etc.) can be analyzed todetermine what classifiers should be used or what weight values shouldbe assigned to their respective classifiers for the purpose ofclassifying media content types. Additionally, multiple classifiers canbe combined to increase their strength for correctly classifying mediacontent types. For example, facial recognition can be combined with thepresence of speech in a combined classifier to identify a voice-over(e.g., speech with no one on screen, etc.), which may be used as afactor to determine whether a particular content portion is a secondarymedia content portion.

In some embodiments, if a media content portion is determined to be asecondary media content portion (e.g., a commercial, etc.), then (e.g.,multi-audio-channel) volume levels or loudness settings of the secondarymedia content portion can be determined, tagged or even adjusted tocomply with legal and/or business requirements. If a media contentportion is determined to be not a secondary media content portion (e.g.,not a commercial, etc.), the media content portion may retain originalvolume levels or loudness settings, for example, as intended by movieproducers, broadcasters, media artists, etc.

In some embodiments, the media fingerprint archive system and the mediameasurement system may be deployed at the same geographic location. Insome other embodiments, the media fingerprint archive system and themedia measurement system may be deployed at different geographiclocations. The systems may communicate or access media fingerprintidentification services over a cloud of networks. In an exampleembodiment, the media measurement system may be located at an operatorfacility, whereas the media fingerprint archive system that supportsonline accesses and provides online fingerprint identification servicesto the media measurement system may be located at a commercialaggregation facility.

As one possible non-limiting application, techniques as described hereincan be used to perform spot checks on media programs, as mandated by theCommercial Advertisement Loudness Mitigation (CALM) Act. Thesetechniques can be used to perform spot checks on a media network oroperator's media programs on 100% of its channels with a large number of(e.g., 50 million, 200 million, etc.) possible commercials for anylength of time.

Instead of capturing hours of programming containing a large amount ofprimary program materials mixed with commercial content, manuallymonitoring/identifying periods of the commercial content in theprogramming, and measuring the commercial content with significant userintervention/involvement as in other approaches, the techniques asdescribed herein provide automatic and reliable detection, measurementand control of commercial content in a media network or operator's mediaprograms, without any need to carry in-band data that identifiessecondary media content portions. While techniques as described hereincan work in the presence of the in-band data, carrying such in-band datacan often add bit overhead requirements to an encoded media signal.Additionally, such in-band data might be used to easily identifycommercials, thereby facilitating commercial-bypassing techniques tobypass those commercials in home viewing environments (which is bad forcontent providers).

Techniques as described herein can be deployed in a wide range of mediasystems, from that of a very large media network/operator/station tothat of a head-end in a local cable network or to that of a smalloff-the-air broadcast station. A media measurement system as describedherein can be provided as a dedicated system, or a module incorporatedinto a media system that has other functions other than mediameasurements, or a media measurement application running on one or morecomputing devices. Techniques can be implemented and/or released assoftware, hardware, firmware, or a combination of two or more of theforegoing. Third-party system providers can also be provided withtoolkits (e.g., System Development Kits or SDKs, etc.) to implementtechniques as described herein in their respective systems.

In some embodiments, mechanisms as described herein form a part of amedia processing system, including but not limited to: aggregationsystem, distribution system, receiving system, broadcast network system,broadcast station system, broadcast operator system, cable broadcastsystem, satellite broadcast system, over-the-air broadcast system, cabledistribution system, satellite distribution system, over-the-airtransceiver system, cable receiving system, satellite receiving system,over-the-air receiving system, head end, desktop computer, computerworkstation, various other kinds of devices and media processingmodules, etc.

Various modifications to the preferred embodiments and the genericprinciples and features described herein will be readily apparent tothose skilled in the art. Thus, the disclosure is not intended to belimited to the embodiments shown, but is to be accorded the widest scopeconsistent with the principles and features described herein.

2. MEDIA DATA AND MEDIA CONTENT PORTIONS

Media data as described herein comprises media sample data (e.g., videoframes, audio frames or PCM audio samples containing media content,etc.) that represents media content portions. A media content portionmay be described herein with reference to one or more example media,including still images, video, and/or audio media. The selection ofexample mediums in this description may be made for simplicity andconcise unity and, unless expressly stated to the contrary, should notbe construed as limiting an embodiment to a particular medium asembodiments of the present invention are well suited to function witheither still images, audio media, or video media. Furthermore,embodiments of the present invention are well suited to function withimages corresponding to audio and/or video media, which may representtwo or three spatial dimensions.

In some embodiments, media data comprises one or more primary mediacontent portions—represented by normal program materials (e.g., apopular TV show, a news program, a live game broadcast, not commercialadvertisements, etc.)—embedded with one or more secondary media contentportions represented by commercial advertisements (or simply“commercials”). The media data comprising both the primary and secondarymedia content portions may be broadcasted, distributed, downloaded,etc., in the form of one or more media signals, one or more mediachannels or media bitstreams from a media data provider system todownstream systems. The media data provider system may insert or place asecondary media content portion (e.g., a commercial, etc.) into aprimary media content portion (e.g., a normal program material, etc.)within a commercial break in the primary media content portion, forexample, in exchange for a payment. In some embodiments, the media datafurther comprises media metadata (e.g., closed caption data, etc.)separate from the media sample data (e.g., video frames, audio frames orPCM audio samples containing media content, etc.) that represents theprimary and secondary media content portions. In some embodiments, themedia metadata in the media data as described herein may carryinformation that can be analyzed along with the media sample data in themedia data for the purpose of classifying media content; however, insome embodiments, the media metadata does not comprise content typeflags, bits, attributes, etc., that positively identify any embeddedsecondary media content portions. Accordingly, a recipient device cannotuse such content type flags, bits, attributes, etc., to skip or replacesecondary media content portions in broadcast operations, playbackoperations or distribution operations.

Examples of media data provider systems that provide or distribute mediadata as described herein and downstream systems that receive the mediadata include, but are not limited only to any of: broadcast networksystems, broadcast station systems, broadcast operator systems, cablebroadcast systems, satellite broadcast systems, over-the-air broadcastsystems, cable distribution systems, satellite distribution systems,over-the-air transceiver systems, cable receiving systems, satellitereceiving systems, over-the-air receiving systems, head ends, etc.

3. MEDIA FINGERPRINT ARCHIVE SYSTEM

FIG. 1 shows a media fingerprint archive system (100) in accordance withone or more embodiments. As shown in FIG. 1, the system (100) includes aServer-side Fingerprint Derivation Unit (108), a Fingerprint StorageUnit (110), a Fingerprint Search Unit (112), and a Data Repository(114), which is a media fingerprint archive for secondary media contentportions to be embedded with primary media content portions in mediadata.

Each of these components is described below and may be located on thesame device (e.g., a server, mainframe, a set-top box, a computer, aclient system, etc.) or may be located on separate devices coupled by anetwork (e.g., Internet, Intranet, Extranet, Local Area Network (LAN),Wide Area Network (WAN), etc.), with wired and/or wireless segments. Inone or more embodiments, the system 100 is implemented using aclient-server topology. The system (100) itself may be an enterpriseapplication running on one or more servers, and in some embodimentscould be a peer-to-peer system, or resident upon a single computingsystem. In addition, the system (100) is accessible from other machinesusing one or more interfaces, web portals, or any other tool to accessthe system 100. In one or more embodiments, the system (100) isaccessible over a network connection, such as the Internet. Informationand/or services provided by the system (100) may also be stored andaccessed over the network connection.

In an embodiment, the Server-side Fingerprint Derivation Unit 108corresponds to software and/or hardware configured to receive secondarymedia content portions 102 and derive (e.g., extract, generate,determine, compute, etc.) one or more media fingerprints (“fingerprint”used herein interchangeably with “signature”) from secondary mediacontent portions that are to embed within one or more primary mediacontent portions in media data.

A media fingerprint may correspond to a fingerprint already archived inthe Data Repository (114), or a fingerprint to be stored in the DataRepository (114), a query fingerprint that is to be searched for in theData Repository (114), etc. Examples of media fingerprints include butare not limited to: video fingerprints, audio fingerprints, etc. Videofingerprints may be derived from images or frames of a video clip (whichis a media content portion). In some embodiments, a video fingerprintcomprises a low bit rate representation of video content from which itis derived and uniquely corresponds to the video content and can becomputed from characteristic components of the video content to whichthe video fingerprint corresponds; the characteristic componentscomprise one or more of luminance, chrominance, or motion descriptors ofthe video content. Audio fingerprints may be derived from audio data orfrom images with embedded audio information (e.g., spectrograms, etc.)in a media content portion. In some embodiments, an audio fingerprintcan be generated from a particular audio waveform as code that uniquelycorresponds to the particular audio waveform. A media fingerprint is nota logically assigned unique identifier such as a sequence number butrather is content-dependent and derived based on specific media contentin a media data unit. In some embodiments, the same media fingerprint ofa media data unit present in different streams of media data may beindependently obtained from the media content of the media data unit bydifferent systems based on the same fingerprint derivation algorithm. Insome embodiments, the substantially same media fingerprints (e.g., wherea small bit portion of the media fingerprints are different, where alarge number of the most significant bits matches, where only a few ofthe least significant bits do not match, etc.) of a media data unitpresent in different streams of media data may be independently obtainedfrom the media content of the media data unit by different systems basedon the same fingerprint derivation algorithm.

The Server-side Fingerprint Derivation Unit 108 may be configured toderive multiple fingerprints from a single secondary media contentportion (e.g., by using fingerprint derivation techniques with varyinglevels of robustness or sensitivity to changes) in the secondary mediacontent portions (102). Furthermore, the Server-side FingerprintDerivation Unit 108 may also be configured to separately derivefingerprints for different components of a secondary media contentportion in the secondary media content portions (102). For example,audio/video data in secondary media content portions may be separatedinto separate components (e.g., audio components and video components,etc.) using a demultiplexer and fingerprints may be derived separatelyfor each component of the secondary media content portion.

In an embodiment, the Fingerprint Storage Unit (110) corresponds tosoftware and/or hardware configured to receive media fingerprintsderived from secondary media content portions from one or morefingerprint sources and store the media fingerprints in the DataRepository (114). In an example, the Fingerprint Storage Unit (110) canreceive media fingerprints derived from secondary media content portions(102) from the Server-side Fingerprint Derivation Unit (108). In anotherexample, the Fingerprint Storage Unit (110) can receive mediafingerprints derived from secondary media content portions from adifferent fingerprint source other than the Server-side FingerprintDerivation Unit (108). In some embodiments, a client (e.g., 200 of FIG.2, etc.) of the system (100) can detect secondary media content portionswhose fingerprints have not been previously archived in the DataRepository (114). The client can provide a detected secondary mediacontent portion to the system (100), or the Server-side FingerprintDerivation Unit (108) therein; or alternatively derive mediafingerprints (104) for the detected secondary media content portion andsubmit the derived media fingerprints (104) to the system (100), or thefingerprint storage (110), for archiving in the Data Repository (114). Amultitude of other implementations (e.g., data structures, relationaltables, lookup tables, hash tables, etc.) may be used for organizingand/or storing the fingerprint in the Data Repository (114).

In an embodiment, the Fingerprint Search Unit (112) corresponds tosoftware and/or hardware configured to receive one or more queryfingerprints (106) in one or more fingerprint identification queriesfrom a client of the system (100) and search for an archived fingerprintthat is derived from a secondary media content portion and archived inthe Data Repository (114) based on the query fingerprint (106). Forexample, archived fingerprints in the Data Repository (114) may belooked up and/or compared to each of the query fingerprints (106) by theFingerprint Search Unit (112) to identify whether a query fingerprintmatches an archived fingerprint. In response to identifying that a queryfingerprint matches an archived fingerprint (e.g., finding an exactmatch, finding an approximate match where a small portion of the queryfingerprint and the archived fingerprint being matched are different,finding an approximate match where a large number of the mostsignificant bits matches, finding an approximate match where only a fewof the least significant bits do not match, etc.), a positivefingerprint identification response for the query fingerprint may besent by the Fingerprint Search Unit (112) to the client that sends afingerprint identification query with the query fingerprint. In responseto identifying that a query fingerprint does not match any archivedfingerprint (e.g., no exact match, finding no approximate match where asmall portion of the query fingerprint and an archived fingerprint beingmatched are different, finding no approximate match where a large numberof the most significant bits matches, finding no approximate match whereonly a few of the least significant bits does not match, etc.), anegative fingerprint identification response for the query fingerprintmay be sent by the Fingerprint Search Unit (112) to the client thatsends a fingerprint identification query with the query fingerprint. Insome embodiments, a fingerprint identification query may includeadditional query information other than a query fingerprint includingbut not limited to identification information for media data from whichthe query fingerprint is derived, etc. In some embodiments, afingerprint identification response may include additional responseinformation other than an indication whether the response is positive ornegative including but not limited to identification information for asecondary media content portion from which the matched archivedfingerprint was derived.

In one or more embodiments of the invention, the Data Repository (114)corresponds to any data storage device (e.g., local memory on a clientmachine, multiple servers connected over the internet, systems within alocal area network, a memory on a mobile device, etc.) or database inwhich media content fingerprints derived from secondary media contentportions may be stored and/or queried for based on query fingerprintsreceived from clients of the media fingerprint archive system (100). Inone or more embodiments of the invention, access to the Data Repository(114) may be restricted and/or secured. As such, access to the DataRepository (114) may require authentication using passwords, secretquestions, personal identification numbers (PINs), biometrics, and/orany other suitable authentication mechanism. Elements or variousportions of data stored in the Data Repository (114) may be distributedand stored in multiple data repositories (e.g., servers across theworld, etc.). In one or more embodiments of the invention, the DataRepository (114) includes flat, hierarchical, network based, relational,dimensional, object modeled, or data files structured otherwise. Forexample, Data Repository (114) may be maintained as a table of a SQLdatabase. In addition, data in the Data Repository (114) may be verifiedagainst data stored in other repositories.

4. MEDIA MEASUREMENT SYSTEM

FIG. 2 shows a media measurement system (200) in accordance with one ormore embodiments. As shown in FIG. 1, the system (200) includes aClient-side Fingerprint Derivation Unit (208), a Media ContentDetermination Unit (210), a Fingerprint Query Unit (212), and a MediaMeasurement Unit (214).

Each of these components is described below and may be located on thesame device (e.g., a server, mainframe, a set-top box, a computer, aclient system, etc.) or may be located on separate devices coupled by anetwork (e.g., Internet, Intranet, Extranet, LAN, WAN, etc.), with wiredand/or wireless segments. In one or more embodiments, the system 200 isimplemented using a client-server topology. The system (200) itself maybe an enterprise application running on one or more servers, and in someembodiments could be a peer-to-peer system, or resident upon a singlecomputing system. In addition, the system (200) is accessible from othermachines using one or more interfaces, web portals, or any other tool toaccess the system 200. In one or more embodiments, the system (200) isaccessible over a network connection, such as the Internet. Informationand/or services provided by the system (200) may also be stored andaccessed over the network connection.

In an embodiment, the Client-side Fingerprint Derivation Unit 208corresponds to software and/or hardware configured to receive media data202 and derive (e.g., extract, generate, determine, compute, etc.) oneor more media query fingerprints (or simply “query fingerprints”) fromthe media data (202) that comprises primary media content portionsembedded with secondary media content portions.

A query fingerprint as generated by the Client-side FingerprintDerivation Unit (208) can be sent to a media fingerprint archive system(e.g., 100 of FIG. 1) and searched for in the database (114). A queryfingerprint may be derived from a secondary media content portion or aprimary media content portion. The Client-side Fingerprint DerivationUnit 108 may be configured to derive multiple fingerprints from a singlemedia content portion (e.g., by using fingerprint derivation techniqueswith varying levels of robustness or sensitivity to changes) in themedia data (202). Furthermore, the Client-side Fingerprint DerivationUnit 208 may also be configured to separately derive fingerprints fordifferent components of a media content portion in the media data (202).For example, audio/video data in the media data (202) may be separatedinto separate components (e.g., audio components and video components)using a demultiplexer and query fingerprints may be derived separatelyfor each component of the media data (202).

In an embodiment, the Fingerprint Query Unit (212) corresponds tosoftware and/or hardware configured to send one or more fingerprintidentification queries (206) with one or more query fingerprints (e.g.,106 of FIG. 1, etc.) to a media fingerprint archive system (e.g., 100 ofFIG. 1, etc.) and receive one or more fingerprint identificationresponses from the media fingerprint archive system (100) to the one ormore fingerprint identification queries (206).

In an embodiment, the Fingerprint Content Determination Unit (210)corresponds to software and/or hardware configured to analyze a mediadata unit in the media data (202) to identify media characteristics ofthe media data unit, and to determine whether the media data unitcorresponds to a secondary media content portion embedded within themedia data (202) based on the identified media characteristics of themedia data unit. In some embodiments, the Fingerprint ContentDetermination Unit (210) is configured to determine whether a media dataunit corresponds to a secondary media content portion embedded withinthe media data (202) in response to the Fingerprint Query Unit (212)receiving a negative fingerprint identification response with respect toa query fingerprint extracted or derived from the media data unit. Insome embodiments, the Fingerprint Content Determination Unit (210) isconfigured to determine whether a media data unit corresponds to asecondary media content portion embedded within the media data (202),regardless of whether the Fingerprint Query Unit (212) receives anegative fingerprint identification response with respect to a queryfingerprint extracted or derived from the media data unit. In someembodiments, the media fingerprint measurement system (200) isconfigured to use one or both of the Fingerprint Query Unit (212) andthe Fingerprint Content Determination Unit (210) to determine whether amedia data unit corresponds to a secondary media content portionembedded within the media data (202).

Media characteristics generally represent any characteristics of mediacontent that may be detected using any audio/video analysis methods.Media characteristics may include visually perceptible characteristicswithin the media content that may be detected. Examples of such visuallyperceptible characteristics that may identify media content include:network watermarks, rating watermarks, captioned/embedded texts,applause, program and scene transition, loudness, etc. Mediacharacteristics may also include other characteristics of the mediacontent that may be detected, such as geographic location, time of day,etc.

In some embodiments, the Media Content Determination Unit (210) furthercomprises one or more classifiers and/or one or more classificationalgorithms configured to classify a media data unit extracted/decodedfrom the media data (202) based on media characteristics identified inthe media data unit. In an embodiment, a classifier or classificationalgorithm assigns individual weight factors to different mediacharacteristics. A weight factor for a media characteristic may scalewith a likelihood of the media data unit being a part of a secondarymedia content portion given the presence of the media characteristic. Inan embodiment, the Media Content Determination Unit (210) uses multipleclassifiers or multiple classification algorithms. The Media ContentDetermination Unit (210) may assign individual weight factors toindividual classifiers or classification algorithms.

In some embodiments, media characteristics identified from a trainingset of media data units may form a set of vectors to determine whether aclassifier has a high error rate or a low error rate in predictingcorrect content types of media data units and to assign weight factorsaccordingly. In some embodiments, a strong classifier may be createdbased on the set of vectors obtained from the training set using an“Adaboost” algorithm. The strong classifier may thereafter be applied toeach media data unit of the media data (202), for the purpose ofdetermining whether the media data unit corresponds to a secondary mediacontent portion or a primary media content portion, and/or for thepurpose of determining whether the media data unit corresponds to a newsecondary media content portion or an existing secondary media contentportion. In an embodiment, as a part of or in addition to classifiers orclassification algorithms, the Media Content Determination Unit (210)also uses algorithms to detect scene changes in the media data (202)based on the (time-dependent) identified media characteristics.

In some embodiments, the Media Content Determination Unit (210) furthercomprises one or more classifiers and/or one or more classificationalgorithms configured to provide a detected secondary media contentportion to a media fingerprint archive system (e.g., 100 of FIG. 1,etc.); or alternatively derive media fingerprints for the detectedsecondary media content portion and submit the derived mediafingerprints (e.g., 104 of FIG. 1, etc.) to a media fingerprint archivesystem (e.g., 100 of FIG. 1, etc.) for archiving in a fingerprintarchive (e.g., the Data Repository 114 of FIG. 1, etc.). Additionalinformation of the secondary media content portion includingcharacteristics such as loudness, etc., may be provided to the mediafingerprint archive system.

In an example, the Media Measurement Unit (214) corresponds to softwareand/or hardware configured to perform one or more measurements over asecondary media content portion in the media data (202). In someembodiments, the Media Measurement Unit (214) is further configured tooutput the measurements performed over the secondary media contentportion in the media data (202) so that data for the measurements ormedia measurement data (204) can be stored, for example, persistently ina media measurement data repository (not shown). The data formeasurements performed over a secondary media content portion (or simply“media measurement data”) includes but is not limited to, any of: one ormore of loudness, length, an identifier of the secondary media contentportion, time of day, channel, network address, etc. In someembodiments, the media measurement data may be stored at a mediameasurement data repository collocated with the media measurement system(210). In some embodiments, the media measurement data may be stored ata media measurement data repository remote to the media measurementsystem (210). In some embodiments, the media measurement data may bestored at a media measurement data repository collocated with a mediafingerprint archive system (e.g., 100 of FIG. 1, etc.). One or more keyscan be generated and/or stored along with the media measurement data toallow fast read and/or write access to the media measurement data basedon given values of the one or more keys.

5. ALGORITHM TO DETECT SECONDARY CONTENT PORTIONS

FIG. 3 illustrates an example algorithm or process flow to search forand perform measurements on secondary media content portions in mediadata. In some example embodiments, one or more computing devices orcomponents may perform this process flow.

A media measurement system (e.g., 200 of FIG. 2, etc.) performing theexample algorithm may receive and/or process media data (e.g., 202 ofFIG. 2, etc.) either in real time, in non-real time, or part in realtime and part in non-real time. In some embodiments, the mediameasurement system (200) is configured to logically divide the mediadata (202) into a sequence of media data units. In some embodiments inwhich the media data (202) is received in a real time broadcast, themedia measurement system (200) is configured to process the media dataunits in the same order as the order in which these media data units arereceived in the media data (202). In some embodiments in which the mediadata (202) is received in a non-real time media signal, a media databitstream, a media file, through a media buffer, etc., the mediameasurement system can be configured to process media data units in themedia data (202) either in the same order as or in a different orderfrom the order in which these media data units are received.

In block 302, for the purpose of illustration only, the mediameasurement system (200) derives a query fingerprint from a media dataunit in the sequence of media data units in the media data (202). Insome embodiments, multiple query fingerprints may be derived from thesame media data unit instead of only one query fingerprint. The mediameasurement system (200) sends a fingerprint identification query withthe derived query fingerprint to a media fingerprint archive system(e.g., 100 of FIG. 1, etc.), in order to determine whether the queryfingerprint matches any archived fingerprint for a secondary mediacontent portion in the media fingerprint archive system (100).

In block 304, the media measurement system (200) receives a fingerprintidentification response to the fingerprint identification query, forexample, sent by the media fingerprint archive system (100).

In block 306, the media measurement system (200) determines whether thefingerprint identification response is a positive one indicating a matchfound between the query fingerprint and an archived fingerprint, or anegative one indicating no such match.

In response to receiving a positive fingerprint identification response,the process flow goes to block 308, in which the media measurementsystem (200) further determines whether the media data unit from whichthe query fingerprint was derived represents a new secondary mediacontent portion in the media data (202). The media data unit mayrepresent a part of media sample data for a new secondary media contentportion, which is preceded by either a primary media content portion ora different secondary media content portion. Alternatively, the mediadata unit may represent a part of media sample data for an existingsecondary media content portion for which an existing set of mediameasurements is being performed.

In some embodiments, a positive fingerprint identification response fromthe media fingerprint archive system (100) may comprise a uniqueidentifier of a secondary media content portion from which the archivedfingerprint matching the query fingerprint was derived and archived. Themedia measurement system (200) can keep track of unique identifiersreceived in positive fingerprint identification responses from the mediafingerprint archive system (100) and determine, based at least in parton the unique identifier in the (current) positive fingerprintidentification response, whether the media data unit from which thequery fingerprint was derived is (a start of) a new secondary mediacontent portion, or whether the media data unit is (a start of) a newsecondary media content portion.

In some embodiments, the media measurement system (200) can analyze themedia data unit in the media data (202) to identify mediacharacteristics of the media data unit, and to determine whether themedia data unit corresponds to a new secondary media content portionembedded within the media data (202) based on the identified mediacharacteristics of the media data unit.

In response to determining that the media data unit is (a start of) anew secondary media content portion, the process flow goes to block 310,in which the media measurement system (200) starts a new set of mediameasurements for the new secondary media content portion.

On the other hand, in response to determining that the media data unitis not (a start of) a new secondary media content portion but rather anexisting media content portion for which an existing set of mediameasurements has started, in block 312, the media measurement system(200) continues the existing set of media measurements for the existingmedia content portion. Subsequently, the processing flow from eitherblock 310 or block 312 goes to block 314.

In response to receiving a negative fingerprint identification response,as determined in block 306, the process flow goes to block 316, in whichthe media measurement system (200) identifies media characteristics ofthe media data unit. Subsequently, in block 318, the media measurementsystem (200) determines, based at least in part on the mediacharacteristics of the media data unit identified in block 316, whetherthe media data unit corresponds to a secondary media content portionembedded within the media data (202).

In response to determining that the media data unit corresponds to asecondary media content portion in block 318, the process flow goes toblock 308, in which the media measurement system (200) furtherdetermines whether the media data unit represents a new secondary mediacontent portion in the media data (202).

As previously noted, in some embodiments, the media measurement system(200) can analyze the media data unit in the media data (202) toidentify media characteristics of the media data unit, and to determinewhether the media data unit corresponds to a new secondary media contentportion embedded within the media data (202) based on the identifiedmedia characteristics of the media data unit. In some embodiments, someor all of the characteristics identified in block 316, as appropriate ifavailable, may be used in block 308 to determine whether the media dataunit represents a new secondary media content portion in the media data(202).

In response to determining that the media data unit does not correspondto a secondary media content portion in block 318, the process flow goesto block 320. For example, the media data unit may correspond to aprimary media content portion (e.g., not a commercial advertisement,etc.). In block 320, the media measurement system (200) performs nomedia measurements for the media data unit from which the queryfingerprint was derived. Subsequently, the process flow goes to block314.

In block 314, the media measurement system (200) determines whether themedia data unit represents the last one in the media data (202). If so,the process flow ends. Otherwise, the process flow goes to block 302 tocontinue processing subsequent media data units in the media data (202).

6. EXAMPLE PROCESS FLOW

FIG. 4A and FIG. 4B illustrate example process flows. In someembodiments, one or more computing devices or units may perform theprocess flows.

In block 402 of FIG. 4A, a media measurement system (e.g., 200 of FIG.2, etc.) extracts a plurality of query fingerprints from media data thatcomprises one or more primary media content portions embedded with oneor more secondary media content portions.

In block 404, the media measurement system (200) sends a plurality offingerprint identification queries to a media fingerprint archivesystem. Each fingerprint identification query in the plurality offingerprint identification queries comprises at least one queryfingerprint in the plurality of query fingerprints.

In block 406, the media measurement system (200) receives a plurality offingerprint identification responses from the media fingerprint archivesystem. Each fingerprint identification response in the plurality offingerprint identification responses is responsive to a correspondingfingerprint query in the plurality of fingerprint identificationqueries, and indicates a positive fingerprint identification or anegative fingerprint identification.

In block 408, the media measurement system (200), in response toreceiving at least one positive fingerprint identification response inthe plurality of fingerprint identification responses from the mediafingerprint archive system: identifies a first secondary media contentportion of the one or more secondary media content portions embeddingwithin the one or more primary media content portion; and performs afirst set of media measurements on the first secondary media contentportion of the one or more secondary media content portions embeddingwithin the one or more primary media content portion.

In an embodiment, the media measurement system (200) is furtherconfigured to, in response to receiving at least one negativefingerprint identification in the plurality of fingerprintidentification responses from the media fingerprint archive system,determine whether a media data portion in the media data comprises oneor more specific characteristics; and in response to determining thatthe media data portion in the media data comprises the one or morespecific characteristics, (a) identifying the media data portion as asecondary media content portion of the one or more secondary mediacontent portions embedding within the one or more primary media contentportion and (b) performing a second set of media measurements on thesecondary media content portion of the one or more secondary mediacontent portions embedding within the one or more primary media contentportion.

In an embodiment, the media measurement system (200) is furtherconfigured to control at least one media characteristic of the secondarymedia content portion in response to determining that the media dataportion in the media data comprises the one or more specificcharacteristics. Under other approaches, “real-time” loudness controldevices, for example those as used by broadcasters, cannot make adistinction between commercials and the accompanying programs andgenerally use the same configuration (e.g. aggressiveness, operationalparameters) for both commercials and the accompanying programs. Incontrast, under techniques as described herein, real-time loudnesscontrollers can be configured to use the fingerprinting system todistinguish between commercials and programs in real-time ornear-real-time, then: (a) re-configure themselves differently for eachcontent type (e.g. use aggressive loudness control on commercials, lightloudness control on programs, etc.); and/or (b) switch off loudnesscontrol altogether on the accompanying programs, while still correctingcommercials.

In an embodiment, the at least one media characteristic as mentionedabove comprises loudness.

In an embodiment, the one or more specific characteristics comprise atleast one of: time of day, duration, frequency of scene changes, facialrecognition, figure/form recognition, presence of speech, presence ofmusic, identification of audio transients, visual spectral content,audio spectral content, video effects, ticklers, pre-exiting secondaryprogram identifiers, network watermarks, rating watermarks,captioned/embedded texts, applause, program and scene transition,loudness, etc.

In an embodiment, the media measurement system (200) is furtherconfigured to send the one or more specific characteristics identifiedfrom the secondary media content portion and one or more fingerprintsderived from the secondary media content portion to a media fingerprintarchive for storage.

In an embodiment, the plurality of media fingerprints comprises one ormore of (a) video fingerprints each of which is derived from images orframes, comprises a low bit rate representation of video content fromwhich it is derived and uniquely corresponds to the video content and iscomputed from characteristic components of the video content to whichthe video fingerprint corresponds or (b) audio fingerprints each ofwhich is generated from a particular audio waveform as code thatuniquely corresponds to the particular audio waveform.

In an embodiment, the media data comprises one or more of: audio contentonly, video content only, or both audio content and video content.

In an embodiment, the media data is received in one of an over-the-airbroadcast signal, a cable broadcast signal, a satellite broadcastsignal, a media data bitstream, or a media data file.

In block 452 of FIG. 4B, a media fingerprint archive system (e.g., 100of FIG. 1, etc.) maintains a media fingerprint archive for secondarymedia content portions that are to embed with primary media contentportions.

In block 454, the media fingerprint archive system (100) receives, froma client, a fingerprint identification query with a query fingerprintderived from media data comprising both primary and secondary mediacontent portions.

In block 456, the media fingerprint archive system (100), in response toreceiving the fingerprint identification query with the queryfingerprint, determines whether the query fingerprint matches with anyfingerprint archived in the media fingerprint archive.

In block 458, the media fingerprint archive system (100), in response todetermining that the query fingerprint matches with a fingerprintarchived in the media fingerprint archive, returns a positivefingerprint identification response to the fingerprint identificationquery, wherein the fingerprint archived in the media fingerprint archiveis derived from a corresponding secondary media content portion.

In an embodiment, the media fingerprint archive system (100) is furtherconfigured to, in response to determining that the query fingerprintdoes not match with any fingerprint archived in the media fingerprintarchive, return a negative fingerprint identification response to thefingerprint identification query.

In an embodiment, the media fingerprint archive system (100) is furtherconfigured to receive a media fingerprint storing request with one ormore media fingerprints generated for a secondary media content portionwhose fingerprints has not previously been archived; and store the mediafingerprints into the media fingerprint archive.

In embodiments, a media processing system configured to perform any oneof the methods as described herein.

In embodiments, an apparatus comprising a processor and configured toperform any one of the methods as described herein.

In embodiments, a non-transitory computer readable storage medium,comprising software instructions, which when executed by one or moreprocessors cause performance of any one of the methods as describedherein.

7. IMPLEMENTATION MECHANISMS—HARDWARE OVERVIEW

According to one embodiment, the techniques described herein areimplemented by one or more special-purpose computing devices. Thespecial-purpose computing devices may be hard-wired to perform thetechniques, or may include digital electronic devices such as one ormore application-specific integrated circuits (ASICs) or fieldprogrammable gate arrays (FPGAs) that are persistently programmed toperform the techniques, or may include one or more general purposehardware processors programmed to perform the techniques pursuant toprogram instructions in firmware, memory, other storage, or acombination. Such special-purpose computing devices may also combinecustom hard-wired logic, ASICs, or FPGAs with custom programming toaccomplish the techniques. The special-purpose computing devices may bedesktop computer systems, portable computer systems, handheld devices,networking devices or any other device that incorporates hard-wiredand/or program logic to implement the techniques.

For example, FIG. 5 is a block diagram that illustrates a computersystem 500 upon which an embodiment of the invention may be implemented.Computer system 500 includes a bus 502 or other communication mechanismfor communicating information, and a hardware processor 504 coupled withbus 502 for processing information. Hardware processor 504 may be, forexample, a general purpose microprocessor.

Computer system 500 also includes a main memory 506, such as a randomaccess memory (RAM) or other dynamic storage device, coupled to bus 502for storing information and instructions to be executed by processor504. Main memory 506 also may be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 504. Such instructions, when stored innon-transitory storage media accessible to processor 504, rendercomputer system 500 into a special-purpose machine that isdevice-specific to perform the operations specified in the instructions.

Computer system 500 further includes a read only memory (ROM) 508 orother static storage device coupled to bus 502 for storing staticinformation and instructions for processor 504. A storage device 510,such as a magnetic disk or optical disk, is provided and coupled to bus502 for storing information and instructions.

Computer system 500 may be coupled via bus 502 to a display 512, such asa liquid crystal display (LCD), for displaying information to a computeruser. An input device 514, including alphanumeric and other keys, iscoupled to bus 502 for communicating information and command selectionsto processor 504. Another type of user input device is cursor control516, such as a mouse, a trackball, or cursor direction keys forcommunicating direction information and command selections to processor504 and for controlling cursor movement on display 512. This inputdevice typically has two degrees of freedom in two axes, a first axis(e.g., x) and a second axis (e.g., y), that allows the device to specifypositions in a plane.

Computer system 500 may implement the techniques described herein usingdevice-specific hard-wired logic, one or more ASICs or FPGAs, firmwareand/or program logic which in combination with the computer systemcauses or programs computer system 500 to be a special-purpose machine.According to one embodiment, the techniques herein are performed bycomputer system 500 in response to processor 504 executing one or moresequences of one or more instructions contained in main memory 506. Suchinstructions may be read into main memory 506 from another storagemedium, such as storage device 510. Execution of the sequences ofinstructions contained in main memory 506 causes processor 504 toperform the process steps described herein. In alternative embodiments,hard-wired circuitry may be used in place of or in combination withsoftware instructions.

The term “storage media” as used herein refers to any non-transitorymedia that store data and/or instructions that cause a machine tooperation in a specific fashion. Such storage media may comprisenon-volatile media and/or volatile media. Non-volatile media includes,for example, optical or magnetic disks, such as storage device 510.Volatile media includes dynamic memory, such as main memory 506. Commonforms of storage media include, for example, a floppy disk, a flexibledisk, hard disk, solid state drive, magnetic tape, or any other magneticdata storage medium, a CD-ROM, any other optical data storage medium,any physical medium with patterns of holes, a RAM, a PROM, and EPROM, aFLASH-EPROM, NVRAM, any other memory chip or cartridge.

Storage media is distinct from but may be used in conjunction withtransmission media. Transmission media participates in transferringinformation between storage media. For example, transmission mediaincludes coaxial cables, copper wire and fiber optics, including thewires that comprise bus 502. Transmission media can also take the formof acoustic or light waves, such as those generated during radio-waveand infra-red data communications.

Various forms of media may be involved in carrying one or more sequencesof one or more instructions to processor 504 for execution. For example,the instructions may initially be carried on a magnetic disk or solidstate drive of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to computer system 500 canreceive the data on the telephone line and use an infra-red transmitterto convert the data to an infra-red signal. An infra-red detector canreceive the data carried in the infra-red signal and appropriatecircuitry can place the data on bus 502. Bus 502 carries the data tomain memory 506, from which processor 504 retrieves and executes theinstructions. The instructions received by main memory 506 mayoptionally be stored on storage device 510 either before or afterexecution by processor 504.

Computer system 500 also includes a communication interface 518 coupledto bus 502. Communication interface 518 provides a two-way datacommunication coupling to a network link 520 that is connected to alocal network 522. For example, communication interface 518 may be anintegrated services digital network (ISDN) card, cable modem, satellitemodem, or a modem to provide a data communication connection to acorresponding type of telephone line. As another example, communicationinterface 518 may be a local area network (LAN) card to provide a datacommunication connection to a compatible LAN. Wireless links may also beimplemented. In any such implementation, communication interface 518sends and receives electrical, electromagnetic or optical signals thatcarry digital data streams representing various types of information.

Network link 520 typically provides data communication through one ormore networks to other data devices. For example, network link 520 mayprovide a connection through local network 522 to a host computer 524 orto data equipment operated by an Internet Service Provider (ISP) 526.ISP 526 in turn provides data communication services through the worldwide packet data communication network now commonly referred to as the“Internet” 528. Local network 522 and Internet 528 both use electrical,electromagnetic or optical signals that carry digital data streams. Thesignals through the various networks and the signals on network link 520and through communication interface 518, which carry the digital data toand from computer system 500, are example forms of transmission media.

Computer system 500 can send messages and receive data, includingprogram code, through the network(s), network link 520 and communicationinterface 518. In the Internet example, a server 530 might transmit arequested code for an application program through Internet 528, ISP 526,local network 522 and communication interface 518.

The received code may be executed by processor 504 as it is received,and/or stored in storage device 510, or other non-volatile storage forlater execution.

8. EQUIVALENTS, EXTENSIONS, ALTERNATIVES AND MISCELLANEOUS

In the foregoing specification, embodiments of the invention have beendescribed with reference to numerous specific details that may vary fromimplementation to implementation. Thus, the sole and exclusive indicatorof what is the invention, and is intended by the applicants to be theinvention, is the set of claims that issue from this application, in thespecific form in which such claims issue, including any subsequentcorrection. Any definitions expressly set forth herein for termscontained in such claims shall govern the meaning of such terms as usedin the claims. Hence, no limitation, element, property, feature,advantage or attribute that is not expressly recited in a claim shouldlimit the scope of such claim in any way. The specification and drawingsare, accordingly, to be regarded in an illustrative rather than arestrictive sense.

What is claimed is:
 1. A method comprising: receiving media data havinga plurality of media data portions; based on media characteristics ofeach media data portion in the plurality of media data portions,determining whether there is a secondary media data portion among theplurality of media data portions; in response to determining that thereis the secondary media data portion among the plurality of media dataportions, causing a loudness controller to adjust a loudness level ofthe secondary media data portion to a different loudness level.
 2. Themethod of claim 1, wherein the media characteristics of the secondarymedia data portion include the loudness level of the secondary mediadata portion.
 3. The method of claim 1, further comprising enabling theloudness controller to distinguish the secondary media data portion fromother media data portions in the plurality of media data portions inreal time or in near real time.
 4. The method of claim 1, furthercomprising enabling the loudness controller to re-configure loudnesscontrol differently for different content types in the plurality ofmedia data portions.
 5. The method of claim 1, further comprisingenabling the loudness controller to switch off loudness control on othermedia data portions in the plurality of media data portions whilecorrecting the loudness level of the secondary media data portion. 6.The method of claim 1, wherein the secondary media data portionrepresents an embedded media data portion in one or more primary mediadata portions in the plurality of media data portions, wherein the oneor more primary media data portions represent a part of a media program,and wherein the secondary media data portion represents no part of themedia program.
 7. The method of claim 1, wherein the media datarepresents a bitstream.
 8. The method of claim 1, wherein media datarepresents one of: audio data only, or audiovisual data.
 9. The methodof claim 1, wherein the loudness controller represents a device used bya media content provider to control loudness settings of secondary mediadata portions inserted into timeslots in media programs.
 10. Anon-transitory computer readable storage medium, storing computerinstructions, which when executed by one or more processors causeperforming: receiving media data having a plurality of media dataportions; based on media characteristics of each media data portion inthe plurality of media data portions, determining whether there is asecondary media data portion among the plurality of media data portions;in response to determining that there is the secondary media dataportion among the plurality of media data portions, causing a loudnesscontroller to adjust a loudness level of the secondary media dataportion to a different loudness level.
 11. The medium of claim 10,wherein the media characteristics of the secondary media data portioninclude the loudness level of the secondary media data portion.
 12. Themedium of claim 10, wherein the computer instructions comprise furtherinstructions, which when executed by the one or more processors causeperforming: enabling the loudness controller to distinguish thesecondary media data portion from other media data portions in theplurality of media data portions in real time or in near real time. 13.The medium of claim 10, wherein the computer instructions comprisefurther instructions, which when executed by the one or more processorscause performing: enabling the loudness controller to re-configureloudness control differently for different content types in theplurality of media data portions.
 14. The medium of claim 10, whereinthe computer instructions comprise further instructions, which whenexecuted by the one or more processors cause performing: enabling theloudness controller to switch off loudness control on other media dataportions in the plurality of media data portions while correcting theloudness level of the secondary media data portion.
 15. The medium ofclaim 10, wherein the secondary media data portion represents anembedded media data portion in one or more primary media data portionsin the plurality of media data portions, wherein the one or more primarymedia data portions represent a part of a media program, and wherein thesecondary media data portion represents no part of the media program.16. The medium of claim 10, wherein the media data represents abitstream.
 17. The medium of claim 10, wherein media data represents oneof: audio data only, or audiovisual data.
 18. The medium of claim 10,wherein the loudness controller represents a device used by a mediacontent provider to control loudness settings of secondary media dataportions inserted into timeslots in media programs.
 19. An apparatuscomprising: one or more processors; a non-transitory computer readablestorage medium, storing computer instructions, which when executed bythe one or more processors cause performing: receiving media data havinga plurality of media data portions; based on media characteristics ofeach media data portion in the plurality of media data portions,determining whether there is a secondary media data portion among theplurality of media data portions; in response to determining that thereis the secondary media data portion among the plurality of media dataportions, causing a loudness controller to adjust a loudness level ofthe secondary media data portion to a different loudness level.
 20. Theapparatus of claim 19, wherein the media characteristics of thesecondary media data portion include the loudness level of the secondarymedia data portion.
 21. The apparatus of claim 19, wherein the computerinstructions comprise further instructions, which when executed by theone or more processors cause performing: enabling the loudnesscontroller to distinguish the secondary media data portion from othermedia data portions in the plurality of media data portions in real timeor in near real time.
 22. The apparatus of claim 19, wherein thecomputer instructions comprise further instructions, which when executedby the one or more processors cause performing: enabling the loudnesscontroller to re-configure loudness control differently for differentcontent types in the plurality of media data portions.
 23. The apparatusof claim 19, wherein the computer instructions comprise furtherinstructions, which when executed by the one or more processors causeperforming: enabling the loudness controller to switch off loudnesscontrol on other media data portions in the plurality of media dataportions while correcting the loudness level of the secondary media dataportion.
 24. The apparatus of claim 19, wherein the secondary media dataportion represents an embedded media data portion in one or more primarymedia data portions in the plurality of media data portions, wherein theone or more primary media data portions represent a part of a mediaprogram, and wherein the secondary media data portion represents no partof the media program.
 25. The apparatus of claim 19, wherein the mediadata represents a bitstream.
 26. The apparatus of claim 19, whereinmedia data represents one of: audio data only, or audiovisual data. 27.The apparatus of claim 19, wherein the loudness controller represents adevice used by a media content provider to control loudness settings ofsecondary media data portions inserted into timeslots in media programs.